Random Vector Research Articles

A MeanShift-guided oversampling with self-adaptive sizes for imbalanced data classification

The imbalanced data classification has gained popularity in machine learning research domain due to its prevalence in numerous applications and its difficulty. However, the majority of contemporary work primarily focuses on addressing between-class imbalance issues. Previous researches have shown that combined with other elements, such as within-class imbalance, small sample size and the presence of small disjuncts, the imbalanced data significantly increase the difficulties for the traditional classifiers to learn. Therefore, we propose a novel MeanShift-guided oversampling with self-adaptive sizes for imbalanced data classification. The proposed MeanShift-guided oversampling technique can simultaneously consider the distribution of minority class and majority class within the sphere with the current minority instance as its center, which can favor addressing small sample size and avoiding overlapping issues often caused by the nearest neighbor (NN)-based oversampling techniques. The incorporation of random vector and flexible cut-off mechanism for vector length can enhance the diversity among the generated synthetic minority instances and avoid overlapping, which makes it suitable for small sample size and small disjuncts problems. To address between-class and within-class imbalance issues, we also introduce a self-adaptive sizes assignment strategy for each minority instance to be oversampled, where the assigned size is inversely proportional to its density and its distance from the majority class. In addition to eliminating within-class imbalance, the strategy can ensure that the informative border minority instances have more opportunities to be oversampled, thus improving classification performance. Extensive experimental results on some datasets with different distributions and imbalance ratios show the proposed algorithm outperforms other compared ones with significant difference.

Recurrent neural networks-guided vector-valued synchronized key exchange for secure and privacy-preserving communication in Industrial Internet of Things

This paper presents an innovative approach to the problem of key exchange in the Industrial Internet of Things (IIoT) implementation. Recurrent Neural Networks (RNNs) and vector-valued neural synchronization are the key components of the proposed secure transmission method. Drive-response methods are integrated to improve speed in important applications, meeting the continuous need for effective cryptographic key exchange between IIoT devices. Conventional algorithms face drawn-out assessment procedures that jeopardize neural synchronization concealment. This work examines how random input vector generation and synchronization issues in RNNs with drive-response are affected by proportional and non-proportional delays. Furthermore, it delves into an unexplored domain, investigating the synchronization of response-based RNN systems without delays and drive-response-based RNN systems with various proportional delays. It also suggests a simplified analysis of Artificial Neural Networks (ANNs) synchronization, organizing ANNs for session key switch-over using an RNN system. To achieve polynomial and non-polynomial synchronization in the proposed driver response systems, this method offers several benefits: it introduces a polynomial synchronization theory for RNNs to generate synchronized input vectors for ANN synchronization; it uses inequality assessment techniques and Lyapunov formulas to derive relevant control inputs and time-dependent conditions; it establishes the relationship between polynomial and non-polynomial synchronization; and it provides numerical examples demonstrating its effectiveness. It also builds a neural network for generating session keys throughout the IIoT network by aligning vector-valued ANNs in a reciprocal manner. This strategy outperforms previous approaches in the literature, validated through simulations, balances resilience against attacks with minimal computational load, resulting in more effective and resilient industrial applications.

Memory and Communication Efficient Federated Kernel k-Means.

A federated kernel k -means (FedKKM) algorithm is developed in this article to conduct distributed clustering with low memory consumption on user devices. In FedKKM, a federated eigenvector approximation (FEA) algorithm is designed to iteratively determine the low-dimensional approximate vectors of the transformed feature vectors, using only low-dimensional random feature vectors. To maintain high communication efficiency in each iteration of FEA, a communication-efficient Lanczos algorithm (CELA) is further designed in FEA to reduce the communication cost. Based on the low-dimensional approximate vectors, the clustering result is obtained by leveraging a distributed linear k -means algorithm. A theoretical analysis shows that: 1) FEA has a convergence rate of O(1/T) , where T is the number of iterations; 2) the scalability of FedKKM is not affected by the dataset size since the communication cost of FedKKM is independent of the number of users' data; and 3) FedKKM is a (1+ϵ) approximation algorithm. The experimental results show that FedKKM achieves the comparable clustering quality to that of a centralized kernel k -means. Compared with state-of-the-art schemes, FedKKM reduces the memory consumption on user devices by up to 94% and also reduces the communication cost by more than 40%.

Soil data augmentation and model construction based on spectral difference and content difference

Soil analysis makes for developing precision agriculture and monitoring land quality, while the models available for spectroscopy-based chemometrics are constrained by limited samples from small areas. The paper proposed sample expansion and model construction based on spectral difference and content difference, realizing data augmentation and deep learning applied to original samples with limited numbers. The spectral subtraction based on maximum or minimum values exploited the maximum or minimum values to acquire the spectral difference and content difference, which provided a new data form for model construction. Keeping enhanced samples whose spectral difference and content difference were all zero was useful for improving model performance. Augmentation of all data or training data based on maximum or minimum values-based spectral subtraction, which sorted the contents and made them the maximum or minimum values in sequence, achieved sample expansion by the spectral difference and content difference. The model utilized the random vector functional link (RVFL) network, extreme learning machine (ELM), and one-dimensional convolutional neural network (1D CNN), which could predict the content of new samples through ensemble averaging when predicting content difference. The experimental result showed the model of the spectral subtraction based on maximum or minimum values had a similar performance to that of the original samples. Augmentation of all data improved model performance by only RVFL and ELM. Augmentation of training data verified 1D CNN was better than RVFL and ELM. The paper implements a new data augmentation method and applies CNN to original samples with inadequate numbers, which lays the foundation for an improved model and applying spectral preprocessing.

A stacked deep multi-kernel learning framework for blast induced flyrock prediction

Blasting operations are widely and frequently used for rock excavation in Civil and Mining constructions. Flyrock is one of the most important issues induced by blasting operations in open pit mines, and therefore needs to be well predicted in order to identify the safety zone to prevent the potential injuries. For this purpose, 234 sets of blasting data were collected from Sungun Copper Mine site, and a stacked deep multi-kernel learning (SD-MKL) framework was proposed to estimate the blast induced flyrock with confidence accuracy. The proposed model uses the stacking-based representation learning framework (S-RL) to achieve deep learning on small-scale training sets. A multi-kernel learning model (MKL) is used as the base module of S-RL framework, which uses a multi-feature fusion strategy to generate multiple kernels with different kernel length in order to reduce the effort in tuning hyperparameters. In addition, this study further enhanced the predictive capability of SD-MKL by introducing the boosting method into the S-RL framework and hence proposed a boosted SD-MKL model. For comparison purpose, several existing machine learning models were implemented, i.e., kernel ridge regression (KRR), support vector machine (SVM), random forest (RF), gradient boosting decision tree (GBDT), ensemble deep random vector functional link (edRVFL), SD-KRR and SD-SVM. Our experimental results showed that the proposed boosted SD-MKL achieved the best overall performance, with the lowest RMSE of 0.21/1.73, MAE of 0.08/0.78, and the highest VAF of 99.98/99.24.

A Modified SVPWM Strategy for Reducing PWM Voltage Noise and Balancing Neutral Point Potential

PWM (pulse width modulation) is the most widely applied current conversion technology, but the high-frequency harmonics it causes have a significant negative impact on inverter system performance. This paper focuses on the three-phase T-type three-level inverter as the research object and addresses existing PWM voltage noise and midpoint potential imbalance issues by proposing an improved random SVPWM strategy, named Neutral Point Potential Balance Random Space Vector PWM (NPB–RSVPWM). The NPB–RSVPWM strategy includes three main steps: (1) introducing a midpoint potential balancing control loop to adjust the synthesis timing of the effective vectors to generate pulse signals, optimizing midpoint potential balance; (2) employing a randomly varying carrier frequency in place of the carrier used in the SVPWM strategy to generate the driving signals for switching devices; and (3) controlling the inverter through the driving pulse signals. This strategy optimizes the synthesis sequence of traditional SVPWM strategy vectors and incorporates random frequency modulation techniques. The mathematical model analyzes PWM harmonic expressions corresponding to fixed switching frequencies, and a random frequency carrier is chosen to suppress these PWM harmonics. The effective vector’s equivalent circuit is analyzed, proposing a technique for optimized vector synthesis timing. The simulation and experimental results verify that the NPB–RSVPWM technique can disperse PWM harmonic energy, reduce voltage noise, and optimize midpoint potential balance. Under the NPB–RSVPWM strategy, the line voltage spectrum becomes uniform, the maximum harmonic content is greatly reduced, and the fluctuation in the DC side midpoint potential is significantly improved. Compared with the traditional SVPWM strategy and random PWM strategy, the NPB–RSVPWM strategy has a lower voltage noise, smaller total harmonic distortion, and a more stable midpoint potential. The effectiveness and feasibility of the NPB–RSVPWM strategy are verified by simulation and experimental results.

A selective LVQ algorithm for improving instance reduction techniques and its application for text classification

Instance-Based Learning, such as the k Nearest Neighbor (kNN), offers a straightforward and effective solution for text classification. However, as a lazy learner, kNN’s performance heavily relies on the quality and quantity of training instances, often leading to time and space inefficiencies. This challenge has spurred the development of instance-reduction techniques aimed at retaining essential instances and discarding redundant ones. While such trimming optimizes computational demands, it might adversely affect classification accuracy. This study introduces the novel Selective Learning Vector Quantization (SLVQ) algorithm, specifically designed to enhance the performance of datasets reduced through such techniques. Unlike traditional LVQ algorithms that employ random vector weights (codebook vectors), SLVQ utilizes instances selected by the reduction algorithm as the initial weight vectors. Importantly, as these instances often contain nominal values, SLVQ modifies the distances between these nominal values, rather than modifying the values themselves, aiming to improve their representation of the training set. This approach is crucial because nominal attributes are common in real-world datasets and require effective distance measures, such as the Value Difference Measure (VDM), to handle them properly. Therefore, SLVQ adjusts the VDM distances between nominal values, instead of altering the attribute values of the codebook vectors. Hence, the innovation of the SLVQ approach lies in its integration of instance reduction techniques for selecting initial codebook vectors and its effective handling of nominal attributes. Our experiments, conducted on 17 text classification datasets with four different instance reduction algorithms, confirm SLVQ’s effectiveness. It significantly enhances the kNN’s classification accuracy of reduced datasets. In our empirical study, the SLVQ method improved the performance of these datasets, achieving average classification accuracies of 82.55%, 84.07%, 78.54%, and 83.18%, compared to the average accuracies of 76.25%, 79.62%, 66.54%, and 78.19% achieved by non-fine-tuned datasets, respectively.

Application of machine learning and multivariate approaches for assessing microplastic pollution and its associated risks in the urban outdoor environment of Bangladesh

Microplastics (MPs) are an emerging global concern due to severe toxicological risks for ecosystems and public health. Therefore, this is the first study in Bangladesh to assess MP pollution and its associated risks for ecosystems and human health in the outdoor urban environment using machine learning and multivariate approaches. The occurrences of MPs in the urban road dust were 52.76 ± 20.24 particles/g with high diversity, where fiber shape (77%), 0.1–0.5 mm size MPs (75%), blue color (26%), and low-density polyethylene (24%) polymer was the dominating MPs category. Pollution load index value (1.28–4.42), showed severe pollution by MPs. Additionally, the contamination factor (1.00–5.02), and Nemerow pollution index (1.38–5.02), indicate moderate to severe MP pollution. The identified polymers based on calculated potential ecological risk (2248.52 ± 1792.79) and polymer hazard index (814.04 ± 346.15) showed very high and high risks, respectively. The occurrences of MPs could effectively be predicted by random forest, and support random vector machine, where EC, salinity, pH, OC, and texture classes were the influencing parameters. Considering the human health aspect, children and adults could be acutely exposed to 19259.68 and 5777.90 MP particles/ year via oral ingestion. Monte-Carlo-based polymers associated cancer risk assessment results indicate moderate risk and high risk for adults and children, respectively, where children were more vulnerable than adults for MP pollution risks. Overall assessment mentioned that Dhaka was the most polluted division among the other divisions.

Random vector functional link networks for function approximation on manifolds

The learning speed of feed-forward neural networks is notoriously slow and has presented a bottleneck in deep learning applications for several decades. For instance, gradient-based learning algorithms, which are used extensively to train neural networks, tend to work slowly when all of the network parameters must be iteratively tuned. To counter this, both researchers and practitioners have tried introducing randomness to reduce the learning requirement. Based on the original construction of Igelnik and Pao, single layer neural-networks with random input-to-hidden layer weights and biases have seen success in practice, but the necessary theoretical justification is lacking. In this study, we begin to fill this theoretical gap. We then extend this result to the non-asymptotic setting using a concentration inequality for Monte-Carlo integral approximations. We provide a (corrected) rigorous proof that the Igelnik and Pao construction is a universal approximator for continuous functions on compact domains, with approximation error squared decaying asymptotically like O(1/n) for the number n of network nodes. We then extend this result to the non-asymptotic setting, proving that one can achieve any desired approximation error with high probability provided n is sufficiently large. We further adapt this randomized neural network architecture to approximate functions on smooth, compact submanifolds of Euclidean space, providing theoretical guarantees in both the asymptotic and non-asymptotic forms. Finally, we illustrate our results on manifolds with numerical experiments.

Advanced forecasting of COVID-19 epidemic: Leveraging ensemble models, advanced optimization, and decomposition techniques

In the global effort to address the outbreak of the new coronavirus pneumonia (COVID-19) pandemic, accurate forecasting of epidemic patterns has become crucial for implementing successful interventions aimed at preventing and controlling the spread of the disease. The correct prediction of the course of COVID-19 outbreaks is a complex and challenging task, mainly because of the significant volatility in the data series related to COVID-19. Previous studies have been limited by the exclusive use of individual forecasting techniques in epidemic modeling, disregarding the integration of diverse prediction procedures. The lack of attention to detail in this situation can yield worse-than-ideal results. Consequently, this study introduces a novel ensemble framework that integrates three machine learning methods (kernel ridge regression (KRidge), Deep random vector functional link (dRVFL), and ridge regression) within a linear relationship (L-KRidge-dRVFL-Ridge). The optimization of this framework is accomplished through a distinctive approach, specifically adaptive differential evolution and particle swarm optimization (A-DEPSO). Moreover, an effective decomposition method, known as time-varying filter empirical mode decomposition (TVF-EMD), is employed to decompose the input variables. A feature selection technique, specifically using the light gradient boosting machine (LGBM), is also implemented to extract the most influential input variables. The daily datasets of COVID-19 collected from two countries, namely Italy and Poland, were used as the experimental examples. Additionally, all models are implemented to forecast COVID-19 at two-time horizons: 10- and 14-day ahead (t+10 and t+14). According to the results, the proposed model can yield higher correlation coefficient (R) for both case studies: Italy (t+10 = 0.965, t+14 = 0.961) and Poland (t+10 = 0.952, t+14 = 0.940) than the other models. The experimental results demonstrate that the model suggested in this paper has outstanding results in various kinds of complex epidemic prediction situations. The proposed ensemble model demonstrates exceptional accuracy and resilience, outperforming all similar models in terms of efficacy.

Bayesian Non-Parametric Inference for Multivariate Peaks-over-Threshold Models.

We consider a constructive definition of the multivariate Pareto that factorizes the random vector into a radial component and an independent angular component. The former follows a univariate Pareto distribution, and the latter is defined on the surface of the positive orthant of the infinity norm unit hypercube. We propose a method for inferring the distribution of the angular component by identifying its support as the limit of the positive orthant of the unit p-norm spheres and introduce a projected gamma family of distributions defined through the normalization of a vector of independent random gammas to the space. This serves to construct a flexible family of distributions obtained as a Dirichlet process mixture of projected gammas. For model assessment, we discuss scoring methods appropriate to distributions on the unit hypercube. In particular, working with the energy score criterion, we develop a kernel metric that produces a proper scoring rule and presents a simulation study to compare different modeling choices using the proposed metric. Using our approach, we describe the dependence structure of extreme values in the integrated vapor transport (IVT), data describing the flow of atmospheric moisture along the coast of California. We find clear but heterogeneous geographical dependence.

Inapproximability of Unique Games in Fixed-Point Logic with Counting

We study the extent to which it is possible to approximate the optimal value of a Unique Games instance in Fixed-Point Logic with Counting (FPC). Formally, we prove lower bounds against the accuracy of FPC-interpretations that map Unique Games instances (encoded as relational structures) to rational numbers giving the approximate fraction of constraints that can be satisfied. We prove two new FPC-inexpressibility results for Unique Games: the existence of a $(1/2, 1/3 + \delta)$-inapproximability gap, and inapproximability to within any constant factor. Previous recent work has established similar FPC-inapproximability results for a small handful of other problems. Our construction builds upon some of these ideas, but contains a novel technique. While most FPC-inexpressibility results are based on variants of the CFI-construction, ours is significantly different. We start with a graph of very large girth and label the edges with random affine vector spaces over $\mathbb{F}_2$ that determine the constraints in the two structures. Duplicator's strategy involves maintaining a partial isomorphism over a minimal tree that spans the pebbled vertices of the graph.

Coherent distributions on the square–extreme points and asymptotics

Abstract Let $\mathcal{C}$ denote the family of all coherent distributions on the unit square $[0,1]^2$ , i.e. all those probability measures $\mu$ for which there exists a random vector $(X,Y)\sim \mu$ , a pair $(\mathcal{G},\mathcal{H})$ of $\sigma$ -fields, and an event E such that $X=\mathbb{P}(E\mid\mathcal{G})$ , $Y=\mathbb{P}(E\mid\mathcal{H})$ almost surely. We examine the set $\mathrm{ext}(\mathcal{C})$ of extreme points of $\mathcal{C}$ and provide its general characterisation. Moreover, we establish several structural properties of finitely-supported elements of $\mathrm{ext}(\mathcal{C})$ . We apply these results to obtain the asymptotic sharp bound $\lim_{\alpha \to \infty}\alpha\cdot(\sup_{(X,Y)\in \mathcal{C}}\mathbb{E}|X-Y|^{\alpha}) = {2}/{\mathrm{e}}$ .

Orthogonal systems based stable estimator for non parametric regression problems

. In this work, we study a univariate as well as a multivariate Jacobi polynomial-based estimator for solving the fundamental problem of non parametric (NP) regression. The proposed NP regression estimator aims to approximate a possibly very low regular regression function from the knowledge of its noised values given at some n random sampling points. We first study the univariate version of the proposed estimator, then extend this study to the general multivariate case with random sampling vectors drawn according to an unknown distribution. Our proposed estimator has the advantage of being stable with reasonable time complexity. Moreover, it has an optimal convergence rate under the weak condition that the regression function has a certain low Sobolev smoothness property. Although the proposed estimators are studied under the hypothesis that the random sampling vectors are supported on the d dimensional unit hypercube, by applying trivial transformations, the results of this work are still valid in the more general setting of any d dimensional hyperrectangle. Finally, the theoretical results of this work are illustrated by some numerical simulations on synthetic as well as real datasets.

On the Submultiplicativity of Matrix Norms Induced by Random Vectors

On the Submultiplicativity of Matrix Norms Induced by Random Vectors

Enhancing short-term wind speed prediction based on an outlier-robust ensemble deep random vector functional link network with AOA-optimized VMD

Wind speed prediction is a crucial aspect in the utilization of wind energy. In this paper, a wind speed prediction model based on an outlier-robust ensemble deep random vector functional link network (ORedRVFL) and arithmetic optimization algorithm-optimized variational mode decomposition (AOA-VMD) is designed. First, the penalty factor and the number of mode decompositions of VMD are optimized using the AOA algorithm and the original data are decomposed using the optimized VMD. Then the decomposed data is predicted using the ensemble deep random vector functional link network (edRVFL) model. The edRVFL uses rich intermediate features for the final decision, which can make the final result closer to the real data. In order to strengthen the anti-interference ability to the outliers, this paper robustly improves the edRVFL model, and the improved model is called ORedRVFL. ORedRVFL reduces the impact of outliers by introducing regularization and norm to balance the relationship between training error and weights. The experiments have proved that the model proposed in this paper outperforms other models in terms of anti-interference ability and prediction accuracy.

Underwater Image Classification Based on EfficientnetB0 and Two-Hidden-Layer Random Vector Functional Link

Underwater Image Classification Based on EfficientnetB0 and Two-Hidden-Layer Random Vector Functional Link

Integrated improved Harris hawks optimization for global and engineering optimization

The original Harris hawks optimization (HHO) algorithm has the problems of unstable optimization effect and easy to fall into stagnation. However, most of the improved HHO algorithms can not effectively improve the ability of the algorithm to jump out of the local optimum. In this regard, an integrated improved HHO (IIHHO) algorithm is proposed. Firstly, the linear transformation escape energy used by the original HHO algorithm is relatively simple and lacks the escape law of the prey in the actual nature. Therefore, intermittent energy regulator is introduced to adjust the energy of Harris hawks, which is conducive to improving the local search ability of the algorithm while restoring the prey's rest mechanism; Secondly, to adjust the uncertainty of random vector, a more regular vector change mechanism is used instead, and the attenuation vector is obtained by modifying the composite function. Finally, the search scope of Levy flight is further clarified, which is conducive to the algorithm jumping out of the local optimum. Finally, in order to modify the calculation limitations caused by the fixed step size, Cardano formula function is introduced to adjust the step size setting and improve the accuracy of the algorithm. First, the performance of IIHHO algorithm is analyzed on the Computational Experimental Competition 2013 (CEC 2013) function test set and compared with seven improved evolutionary algorithms, and the convergence value of the iterative curve obtained is better than most of the improved algorithms, verifying the effectiveness of the proposed IIHHO algorithm. Second, the IIHHO is compared with another three state of the art (SOTA) algorithms with the Computational Experimental Competition 2022 (CEC 2022) function test set, the experiments show that the proposed IIHHO algorithm still has a strong ability to search for the optimal value. Third, IIHHO algorithm is applied in two different engineering experiments. The calculation results of minimum cost prove that IIHHO algorithm has certain advantages in dealing with the problem of search space. All these demonstrate that the proposed IIHHO is promising for numeric optimization and engineering applications.

Augmenting the diversity of imbalanced datasets via multi-vector stochastic exploration oversampling

The problem of class imbalance is prevalent in many real-world data sets, causing learning models to skew towards the majority class and resulting in biased performance. Data augmentation methods, such as the well-known Synthetic Minority Over-sampling Technique (SMOTE), are commonly employed to address class imbalance by generating synthetic samples. However, the generation mechanism of SMOTE is relatively constrained resulting in insufficient diversity in synthetic samples. To overcome this limitation, this paper expands the classical SMOTE and introduces a novel generalized version, namely Multi-vector Stochastic Exploration Oversampling (MSEO). It broadens the set of mapping synthetic samples, originally formed by the determined direction vectors and scaling vectors through the neighboring samples, to a collection obtained through mappings with random direction vectors and scaling vectors. This allows the generated samples to escape the original linear interpolation region, facilitating a more flexible exploration of the sample space. We extensively evaluated the method on various types of datasets, including artificially generated datasets, multi-class real-world datasets, and the engineering dataset. The results indicate that MSEO exhibits significant advantages in enhancing classification performance and promoting diversity in synthetic samples.

On the effect of confounding in linear regression models: an approach based on the theory of quadratic forms

AbstractIn the last two decades, significant research efforts have been dedicated to addressing the issue of spatial confounding in linear regression models. Confounding occurs when the relationship between the covariate and the response variable is influenced by an unmeasured confounder associated with both. This results in biased estimators for the regression coefficients reduced efficiency, and misleading interpretations. This article aims to understand how confounding relates to the parameters of the data generating process. The sampling properties of the regression coefficient estimator are derived as ratios of dependent quadratic forms in Gaussian random variables: this allows us to obtain exact expressions for the marginal bias and variance of the estimator, that were not obtained in previous studies. Moreover, we provide an approximate measure of the marginal bias that gives insights of the main determinants of bias. Applications in the framework of geostatistical and areal data modeling are presented. Particular attention is devoted to the difference between smoothness and variability of random vectors involved in the data generating process. Results indicate that marginal covariance between the covariate and the confounder, along with marginal variability of the covariate, play the most relevant role in determining the magnitude of confounding, as measured by the bias.